Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[20542] Refactor Github CI sanitizer related jobs #4662

Merged
merged 12 commits into from
Apr 10, 2024

Conversation

EduPonz
Copy link

@EduPonz EduPonz commented Apr 5, 2024

Description

This PR refactors the Github CI for Address and Thread Sanitizers. It removes the old workflows and adds a single reusable one that defines 4 jobs:

  • asan_fastdds_build
  • asan_fastdds_test (uses the build from asan_fastdds_build)
  • asan_discovery_server_test (uses the build from asan_fastdds_build)
  • tsan_fastdds_test

Furthermore, it adds both a sanitizers-ci.yaml and nightly-sanitizers-ci.yaml so that:

  • On PR review request and once a day, all job run. For the case of PRs, they only run if there are no conflicts and the no-test and skip-ci labels are not set.
  • Manual dispatching allows for individually selecting whether to run asan_fastdds_test, asan_discovery_server_test, and tsan_fastdds_test. This way it is possible to run a subset of the jobs defined in the workflow.
  • Removes address_sanitizers.yaml
  • Removes thread_sanitizer.yaml

@Mergifyio backport 2.13.x 2.10.x 2.6.x
IMPORTANT: Remove the nightly workflows from the backports, as they will not run anyways.

Contributor Checklist

  • Commit messages follow the project guidelines.
  • N/A: The code follows the style guidelines of this project.
  • N/A: Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally
  • N/A: Any new/modified methods have been properly documented using Doxygen.
  • N/A: Any new configuration API has an equivalent XML API (with the corresponding XSD extension)
  • N/A: Changes are ABI compatible.
  • N/A: Changes are API compatible.
  • N/A: New feature has been added to the versions.md file (if applicable).
  • N/A: New feature has been documented/Current behavior is correctly described in the documentation.
  • Applicable backports have been included in the description.

Reviewer Checklist

  • The PR has a milestone assigned.
  • The title and description correctly express the PR's purpose.
  • Check contributor checklist is correct.
  • Check CI results: changes do not issue any warning.
  • Check CI results: failing tests are unrelated with the changes.

@EduPonz EduPonz requested a review from JesusPoderoso April 5, 2024 15:26
@EduPonz EduPonz force-pushed the ci/refactor_sanitizers branch from a96f890 to 77c793e Compare April 5, 2024 15:28
@EduPonz EduPonz requested review from JesusPoderoso and removed request for JesusPoderoso April 5, 2024 15:28
@EduPonz EduPonz force-pushed the ci/refactor_sanitizers branch 3 times, most recently from 6427c49 to f0cca47 Compare April 5, 2024 15:52
@github-actions github-actions bot added the ci-pending PR which CI is running label Apr 5, 2024
@EduPonz EduPonz force-pushed the ci/refactor_sanitizers branch 11 times, most recently from 2c2dc06 to 78b0e40 Compare April 6, 2024 07:41
@EduPonz EduPonz added this to the v2.14.1 milestone Apr 6, 2024
@EduPonz EduPonz force-pushed the ci/refactor_sanitizers branch 2 times, most recently from f2820b6 to c98b293 Compare April 6, 2024 08:24
@EduPonz EduPonz requested a review from JesusPoderoso April 6, 2024 08:41
@EduPonz EduPonz added the needs-review PR that is ready to be reviewed label Apr 6, 2024
@EduPonz EduPonz requested review from JesusPoderoso and removed request for JesusPoderoso April 6, 2024 12:22
@EduPonz
Copy link
Author

EduPonz commented Apr 6, 2024

@Mergifyio backport 2.13.x 2.10.x 2.6.x

Copy link
Contributor

mergify bot commented Apr 6, 2024

backport 2.13.x 2.10.x 2.6.x

✅ Backports have been created

@EduPonz EduPonz requested review from JesusPoderoso and removed request for JesusPoderoso April 7, 2024 06:17
.github/workflows/sanitizers-ci.yml Outdated Show resolved Hide resolved
.github/workflows/nightly-sanitizers-ci.yml Outdated Show resolved Hide resolved
.github/workflows/reusable-sanitizers-ci.yml Outdated Show resolved Hide resolved
.github/workflows/reusable-sanitizers-ci.yml Outdated Show resolved Hide resolved
.github/workflows/reusable-sanitizers-ci.yml Outdated Show resolved Hide resolved
.github/workflows/reusable-sanitizers-ci.yml Outdated Show resolved Hide resolved
@JesusPoderoso JesusPoderoso removed the needs-review PR that is ready to be reviewed label Apr 8, 2024
@JesusPoderoso JesusPoderoso self-requested a review April 8, 2024 13:13
Copy link
Contributor

@JesusPoderoso JesusPoderoso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with green CI

@JesusPoderoso JesusPoderoso added ready-to-merge Ready to be merged. CI and changes have been reviewed and approved. and removed ci-pending PR which CI is running labels Apr 9, 2024
@EduPonz EduPonz merged commit ac0e737 into master Apr 10, 2024
7 of 12 checks passed
@EduPonz EduPonz deleted the ci/refactor_sanitizers branch April 10, 2024 05:11
mergify bot pushed a commit that referenced this pull request Apr 10, 2024
* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add asan_discovery_server_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add tsan_fastdds_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Create a reusable workflow and a CI one for PRs or manual

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add nightly sanitizer CI for all supported branches

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Change extension to .yml

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove old workflows

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Run sanitizers CI on PR review request

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Exclude xfail tests in discovery server test run

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Install all python packages needed for asan_fastdds_test reporting

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Apply Jesús' suggestions

Signed-off-by: eduponz <[email protected]>

---------

Signed-off-by: eduponz <[email protected]>
(cherry picked from commit ac0e737)

# Conflicts:
#	.github/workflows/address-sanitizers.yaml
#	.github/workflows/thread-sanitizer.yaml
mergify bot pushed a commit that referenced this pull request Apr 10, 2024
* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add asan_discovery_server_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add tsan_fastdds_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Create a reusable workflow and a CI one for PRs or manual

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add nightly sanitizer CI for all supported branches

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Change extension to .yml

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove old workflows

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Run sanitizers CI on PR review request

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Exclude xfail tests in discovery server test run

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Install all python packages needed for asan_fastdds_test reporting

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Apply Jesús' suggestions

Signed-off-by: eduponz <[email protected]>

---------

Signed-off-by: eduponz <[email protected]>
(cherry picked from commit ac0e737)

# Conflicts:
#	.github/workflows/address-sanitizers.yaml
#	.github/workflows/config/asan_colcon.meta
#	.github/workflows/thread-sanitizer.yaml
mergify bot pushed a commit that referenced this pull request Apr 10, 2024
* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add asan_discovery_server_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add tsan_fastdds_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Create a reusable workflow and a CI one for PRs or manual

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add nightly sanitizer CI for all supported branches

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Change extension to .yml

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove old workflows

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Run sanitizers CI on PR review request

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Exclude xfail tests in discovery server test run

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Install all python packages needed for asan_fastdds_test reporting

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Apply Jesús' suggestions

Signed-off-by: eduponz <[email protected]>

---------

Signed-off-by: eduponz <[email protected]>
(cherry picked from commit ac0e737)

# Conflicts:
#	.github/workflows/address-sanitizers.yaml
#	.github/workflows/config/asan_colcon.meta
#	.github/workflows/thread-sanitizer.yaml
JesusPoderoso pushed a commit that referenced this pull request Apr 10, 2024
* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add asan_discovery_server_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add tsan_fastdds_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Create a reusable workflow and a CI one for PRs or manual

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add nightly sanitizer CI for all supported branches

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Change extension to .yml

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove old workflows

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Run sanitizers CI on PR review request

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Exclude xfail tests in discovery server test run

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Install all python packages needed for asan_fastdds_test reporting

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Apply Jesús' suggestions

Signed-off-by: eduponz <[email protected]>

---------

Signed-off-by: eduponz <[email protected]>
(cherry picked from commit ac0e737)
EduPonz added a commit that referenced this pull request Apr 17, 2024
* Refactor Github CI sanitizer related jobs (#4662)

* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add asan_discovery_server_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add tsan_fastdds_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Create a reusable workflow and a CI one for PRs or manual

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add nightly sanitizer CI for all supported branches

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Change extension to .yml

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove old workflows

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Run sanitizers CI on PR review request

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Exclude xfail tests in discovery server test run

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Install all python packages needed for asan_fastdds_test reporting

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Apply Jesús' suggestions

Signed-off-by: eduponz <[email protected]>

---------

Signed-off-by: eduponz <[email protected]>
(cherry picked from commit ac0e737)

# Conflicts:
#	.github/workflows/address-sanitizers.yaml
#	.github/workflows/config/asan_colcon.meta
#	.github/workflows/thread-sanitizer.yaml

* Refs #20703: Fix conflicts

Signed-off-by: Mario Dominguez <[email protected]>

* Refs #20703: Remove sanitizer nightlies

Signed-off-by: Mario Dominguez <[email protected]>

* Refs #20703: Remove old sanitizer jobs

Signed-off-by: Mario Dominguez <[email protected]>

* Refs #20703: set default discovery server branch to v1.2.1

Signed-off-by: Mario Dominguez <[email protected]>

* #20703: Address review changes

Signed-off-by: Mario Dominguez <[email protected]>

---------

Signed-off-by: Mario Dominguez <[email protected]>
Co-authored-by: Eduardo Ponz Segrelles <[email protected]>
Co-authored-by: Mario Dominguez <[email protected]>
EduPonz added a commit that referenced this pull request Apr 18, 2024
* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add asan_discovery_server_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add tsan_fastdds_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Create a reusable workflow and a CI one for PRs or manual

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add nightly sanitizer CI for all supported branches

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Change extension to .yml

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove old workflows

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Run sanitizers CI on PR review request

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Exclude xfail tests in discovery server test run

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Install all python packages needed for asan_fastdds_test reporting

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Apply Jesús' suggestions

Signed-off-by: eduponz <[email protected]>

---------

Signed-off-by: eduponz <[email protected]>
(cherry picked from commit ac0e737)

Co-authored-by: Eduardo Ponz Segrelles <[email protected]>
EduPonz added a commit that referenced this pull request Apr 18, 2024
* Refactor Github CI sanitizer related jobs (#4662)

* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add asan_discovery_server_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add tsan_fastdds_test job

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Create a reusable workflow and a CI one for PRs or manual

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Add nightly sanitizer CI for all supported branches

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Change extension to .yml

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Remove old workflows

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Run sanitizers CI on PR review request

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Exclude xfail tests in discovery server test run

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Install all python packages needed for asan_fastdds_test reporting

Signed-off-by: eduponz <[email protected]>

* Refs #20542: Apply Jesús' suggestions

Signed-off-by: eduponz <[email protected]>

---------

Signed-off-by: eduponz <[email protected]>
(cherry picked from commit ac0e737)

# Conflicts:
#	.github/workflows/address-sanitizers.yaml
#	.github/workflows/config/asan_colcon.meta
#	.github/workflows/thread-sanitizer.yaml

* Fix conflicts

Signed-off-by: JesusPoderoso <[email protected]>

---------

Signed-off-by: JesusPoderoso <[email protected]>
Co-authored-by: Eduardo Ponz Segrelles <[email protected]>
Co-authored-by: JesusPoderoso <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-to-merge Ready to be merged. CI and changes have been reviewed and approved.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants